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(54) Watermarks for detecting tampering in images 



(57) A watermarking scheme f or images which in- 
clude techniques for inserting and extracting water- 
marks in the frequency domain and for determining 
whether an image so watermarked has been tampered 
with. Watermark insertion is accomplished by embed- 
ding the bits jof ajjigital signature of a hash function of 
the image in the .frequency, coefficients of the imafle. 
Tamper detection is accomplished generally as follows: 



the watermark which was embedded during the water- 
mark insertion process is extracted from the image; the 
hash function of the image is computed as in the inser- 
tion process; it is verified using a public key whether the 
extracted watermark is a valid signature of the hash val- 
ue. If so, then there is assurance that the image has not 
been tampered with. Otherwise, there is reason to con- 
clude that the image has been tampered with. 
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Description 

[0001] This invention relates generally to a water- 
marking scheme, and more particularly to techniques tor 
inserting and extracting watermarks directly in the fre- 
quency domain of a compressed image and for deter- 
mining whether an image so watermarked has been 
tampered with. 

[0002] A watermark is a digital pattern inserted into a 
digital creation, such as a digital image. The process of 
inserting a watermark into a digital image can be done 
directly in the frequency domain representation typically 
used in compressed images. The watermark can be in- 
serted by altering certain frequency coefficients while 
minimizing the resulting distortion. In case of a block- 
based frequency domain representation, such as that 
used in the JPEG image compression standard, fre- 
quencies as well as blocks can be chosen to be altered 
so that the distortion is minimized. In either case, after 
the watermark has been inserted into the image, a pro- 
cedure, which may be implemented using certain soft- 
ware, is typically required to validate the watermark. 
[0003] There are different types of watermarks which 
serve different purposes. Tamper- resistant watermarks, 
for example, are designed to identify ownership or the 
intended recipient of a digital image. To function effec- 
tively as an identifier (i.e., to trace unauthorized distri- 
bution of an image), a tamper-resistant watermark must 
be embedded in the image so that it is impossible, or at 
least difficult, to remove the watermark without visibly 
damaging the image. Such a watermark must also be 
resistant to image processing techniques, such as crop- 
ping, scaling, image enhancement, compression/de- 
compression, etc. In addition, a tamper-resistant water- 
mark should be readily detectable and recoverable by 
the proper authorities to permit the tracing and identifi- 
cation of the image, even if someone has tampered with 
the image. 

[0004] Another type of watermark, sometimes re- 
ferred to as fragile watermarks, are designed to detect 
tampering of an image. A fragile watermark is embed- 
ded in a digital image so that, if someone tampers with 
the image, that tampering will modify (or destroy) the 
watermark. Fragile watermarks may be used, for exam- 
ple, in connection with images generated by digital cam- 
eras to provide a basis for determining whether or not 
an image has been tampered with after its creation. 
[0005] Various watermarking methods have been 
proposed. For example, U.S. Patent No. 5,530,759 pro- 
poses a digital watermark applied to an original image 
as a multiplicative correction to pixel sample values of 
the original image in a linear color space such that the 
chromaticities of the pixels are not changed. This pro- 
cedure results in a visible watermark which is simply 
added to the original image. 

[0006] U.S. Patent No. 5,606,609 sets forth an elec- 
tronic document verification system and method. This 
refers to a scheme for electronically signing documents, 



but the signature is not embedded in the document data 
itself (i.e., the document data is not modified). The sig- 
nature is just added to the document as another field. 
[0007] U.S. Patent Nos. 5,613,004 and 5,687,236 

s combine steganography (hiding information that is oth- 
erwise in plain view) and cryptography (scrambling in- 
formation that may be sent by unsecured means). Data 
is watermarked so that if it is copied, it is possible to 
determine who owns the original. Thus, the watermark- 

io ing schemes of these patents are of the tamper-resistant 
type. 

[0008] U.S. Patent 5,664,018 proposes a watermark- 
ing procedure wherein each of a set of copies of a dig- 
itized work has a slightly-modified form of a "baseline" 

is watermark that is placed within a critical region of the 
data. This is a tamper-resistant scheme that makes it 
difficult to remove the watermark without damaging the 
content, even if several parties with differently water- 
marked copies of a single image collude together in an 

20 attempt to remove the watermark. 

[0009] U.S. Patent No. 5,689,587 sets forth a method 
and apparatus for hiding data in images. This is a way 
to hide information in images, again to ensure tamper 
resistance (copyright type protection). 

25 [001 0] However, none of these patents provide a frag- 
ile watermarking scheme for detecting tampering. More- 
over, none of these patents provide a watermarking 
technique which may be implemented directly in the fre- 
quency domain representation of an image such that the 

30 distortion resulting from the embedded watermark is 
minimized. Yet another shortcoming of these patents is 
that they do not provide insertion and verification proce- 
dures that work without having to completely decom- 
press a compressed image. 

35 [0011] Therefore, it is an object of the present inven- 
tion to overcome the aforementioned problems. 
[001 2] It is another object of the present invention to 
provide a scheme for inserting and extracting water- 
marks in frequency domain and for verifying whether an 

40 image so watermarked has been tampered with. 

[001 3] It is still another object of the invention to pro- 
vide a verification procedure in connection with a water- 
marking scheme for images, whereby, if tampering has 
occurred, the verification procedure will reveal a dam- 

45 aged watermark and hence provide a basis for declaring 
that the image has been tampered with. 
[0014] It is yet another object of the invention to pro- 
vide a frequency domain watermark insertion and veri- 
fication process that uses a secret key only in the inser- 

50 tion step while the verification is carried out using a pub- 
licly available key. 

[0015] It is yet another object of the invention to pro- 
vide a watermark insertion and verification process that 
does not require full decompression of a compressed 
55 image. 

[0016] According to one aspect of the invention, a 
technique for embedding a watermark in a digital image 
and a technique for detecting tampering of a digital im- 
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age so watermarked are provided. 
[0017] Another aspect of the invention involves wa- 
termarking a compressed digital image and detecting 
tampering of a compressed image so watermarked, 
without having to completely decompress the image. 
[001 8] Yet another aspect of the invention involves an 
insertion mechanism that uses a secret key while the 
corresponding verification mechanism uses a publicly 
available key. 

[0019] Watermarking the compressed image initially 
involves computing a hash value for the image which is 
accomplished by: partially decoding the compressed 
digital image to generate a plurality of blocks, each block 
having a plurality of transform coefficients; obtaining a 
quantizer of the highest frequency coefficient in each 
block and multiplying that coefficient by its quantizer; de- 
termining whether to embed a watermark bit in each 
block based on the highest frequency transform coeffi- 
cient in that block and the number of watermark bits re- 
maining to be embedded in the digital image; setting to 
zero the least significant bit (LSB) of the highest fre- 
quency transform coefficient in each block for which it 
was determined to embed a watermark bit; updating a 
has value at each block by applying a hash function at 
each block, wherein the hash value computed at the last 
block is a multiple-bit value representative of the entire 
digital image. Once the hash value for the entire image 
is computed, a watermark is computed from the com- 
puted hash value using a secret key and a digital signa- 
ture algorithm. Then, each watermark bit is embedded 
in one of the blocks for which it was previously deter- 
mined to do so by setting the LSB of the highest frequen- 
cy transform coefficient in that block to match the corre- 
sponding watermark bit. 

[0020] To determine whether a compressed image so 
watermarked has been tampered with, the following 
steps are performed: partially decoding the compressed 
watermarked image to generate a plurality of blocks, 
each block having a plurality of transform coefficients; 
determining each block in which a watermark bit is em- 
bedded; extracting, from each block which was previ- 
ously determined to be an embedder of a watermark bit, 
the LSB of the highest frequency transform coefficient 
in that block to generate an extracted watermark; com- 
puting a hash value of the digital image by applying a 
hash function at each block based on the zeroed value 
of the LSB of the highest frequency transform coefficient 
in that block, wherein the hash value computed at the 
last block is a multiple-bit value representative of the en- 
tire digital image; applying a digital signature algorithm 
to the extracted watermark and the computed hash val- 
ue and a public key to determine whether the com- 
pressed watermarked image has been tampered with. 
[0021] The watermark insertion procedures can be 
done directly in an image capturing device, such as a 
digital camera, or can be performed by an appropriately 
configured computer Such a computer can also be used 
to check a watermarked image to determine if tampering 



has occurred and, if tampering has occurred, to deter- 
mine where it occurred. 

[0022] Other objects and attainments together with a 
fuller understanding of the invention will become appar- 
s ent and appreciated by referring to the following descrip- 
tion and claims taken in conjunction with the accompa- 
nying drawings. 

[0023] In the drawings wherein like reference symbols 
refer to like parts: 
10 [0024] Fig. 1 is a diagram generally showing the in- 
sertion of a watermark into an image in frequency do- 
main in accordance with an embodiment of the inven- 
tion. 

[0025] Fig. 2 is a flow diagram depicting an initial scan 
f 5 procedure used in connection with a method of inserting 
a watermark into an image in frequency domain in ac- 
cordance with an embodiment of the invention. 
[0026] Fig. 3 is a flow diagram depicting a procedure 
to embed bits into an image in connection with a method 
of inserting a watermark into an image in frequency do- 
main in accordance with an embodiment of the inven- 
tion. 

[0027] Fig. 4 is a schematic diagram showing the ver- 
ification of the existence of the fragile watermark in fre- 
quency domain from a watermarked image in accord- 
ance with an embodiment of the invention. 
[0028] Fig. 5 is a flow diagram depicting a method of 
extracting the watermark from an image and verifying 
its validity to determine if the image has been tampered 
with. 

[0029] Fig. 6 is a block diagram of a digital camera 
adapted for use in connection with the invention. 
[0030] Fig. 7 is a block diagram that illustrates the in- 
terrelationship between various components that may 
be used in capturing and viewing digital images, as well 
as processing such images in accordance with the in- 
vention. 

[0031] To watermark an image in the frequency do- 
main, the image is scanned once to compute H, which 
represents a k-bit hash value of the image. Using a dig- 
ital signature algorithm, S, and a secret key, K s , the m- 
bit watermark W = S(H,K S ) is computed. A second pass 
of the image is then made to embed the watermark Win- 
to the image. The process is illustrated in Figs. 1, 2 and 
3. 

[0032] Referring primarily to Fig. 2 and secondarily to 
Figs. 1 and 3, in the initial scan procedure, the hash val- 
ue H is initialized to afixed value in step 101. The com- 
pressed image data, which may be in the form of a JPEG 
image I, is supplied to a decoder in step 102. The de- 
coder parses the headers from the JPEG data, noting 
the value of q, the quantization table entry for the highest 
frequency coefficient, which is the 63 rd coefficient for the 
8x8 discrete cosine transform (DCT) used in JPEG. If 
there is another block of coefficients yet to be decoded 
and processed (step 103), the next such block, B jt is 
partially decoded in step 104. Only the entropy coding 
of the compressed data is undone, avoiding the de-zig- 
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zagging, dequantization, and inverse discrete cosine 
transform (IDCT) steps needed for full decompression. 
This results in a representation of B-, made up of only the 
non-zero quantized coefficients along with their loca- 
tions in the zig-zag order. Since we use the 63 rd coeffi- 
cient in special ways, the representation of Bj is always 
made to have the 63 rd coefficient even when its value is 
zero. Note that this can be easily done without de-zig- 
zagging the other non-zero quantized coefficients be- 
cause the 63 rd coefficient is the last coefficient in the zig- 
zag order. The decoded representation of the block B-, 
is then passed to step 105, where the 63 rd coefficient is 
multiplied by its quantizer, q (obtained in step 102). The 
reason for this step is as follows. Small variations in the 
higher frequencies are invisible to the human eye. The 
watermark W is embedded bit-by-bit in the image by 
modifying only the highest frequency coefficient (the 
63 rd coefficient in this case) so as to minimize the dis- 
tortion. A watermark bit is embedded (later, in step 207, 
Fig. 3) in a coefficient value by changing the LSB of the 
value to be equal to the watermark bit. If a quantized 
63 rd coefficient has the value v, then changing its LSB 
changes the dequantized coefficient by plus or minus q, 
where q is the quantizer for the 63 rd coefficient. To min- 
imize this change, we set the quantizer for the 63^ co- 
efficient to be 1 and replace the 63 rd coefficient in every 
block (not just those embedding watermark bits) directly 
by their dequantized values (which are obtained by mul- 
tiplying by q). On decompression, the distortion can only 
be plus or minus 1 , as a result of this step. Since the 
63 rd coefficient is typically zero in most of the blocks, 
the increase in compressed size resulting from the mul- 
tiplication in step 105 is minimal, as only the non-zero 
coefficients account for most of the compressed size. 
[0033] A decision is made as to whether a watermark 
bit is to be embedded in block Bi in step 106. The deci- 
sion procedure for step 106 is designed so as to mini- 
mize the resulting distortion in the image as well as to 
minimize the resulting size-increase of the compressed 
image. We denote this decision procedure by the name 
EMBEDDER-TEST, to simplify the subsequent presen- 
tation as this decision procedure is used again in two 
places. EMBEDDER-TEST is fully described as follows. 
[0034] For color images, watermark bits are embed- 
ded only in the luminance plane of the image. This is 
done so that during decompression, when the lumi- 
nance-chrominances color representation is converted 
back to red, green, and blue pixel values (RGB), the re- 
sulting distortion is minimized. Moreover, the chromi- 
nance planes are typically subsampled, so any distor- 
tion in a single chrominance block results in distortions 
in several RGB blocks. Thus, in grayscale images as 
well as in color images, watermark bits are embedded 
only in the color component numbered zero (which is 
the luminance plane for color images). To minimize the 
distortion, watermark bits are embedded only in the 63^ 
DCT coefficient, as mentioned previously. To minimize 
the compressed size, only those blocks are chosen to 



6 

embed a watermark bit where the 63 rd coefficient is al- 
ready non-zero. This follows from the observation that 
changing a zero value to a non-zero value results in a 
far greater increase in compressed size, compared to 

s changing a non-zero value to another non-zero value. 
However, since EMBEDDER-TEST will also be per- 
formed by the watermark verification procedure, we do 
not choose as embedders' even the blocks where the 
63 rd coefficient (dequantized) is plus or minus 1, as it 

10 might potentially be turned to zero on embedding the 
watermark bit, and then the verifier will not be able to 
decide the block to be an embedder. If, at some point, 
the number of watermark bits remaining to be embed- 
ded becomes equal to the number of blocks remaining 

is in component zero, every subsequent block in compo- 
nent zero is decided upon as an embedder of a water- 
mark bit. 

[0035] If the decision to embed a bit in block Bi is yes, 
then the LSB of the 63 rd DCT coefficient is set to zero 
20 in step 107 and the procedure proceeds to step 108. If 
the decision is no, then the procedure directly proceeds 
to step 108. The hash value H is updated in step 108 
using its previous value, and the values of all the non- 
zero quantized coefficients in Bj along with their loca- 
ls tions in the zig-zag order, using a one-way hash func- 
tion. The 63 rd coefficient is always included in this com- 
putation even if it is zero. 

[0036] When all the blocks have been processed, the 
procedure proceeds to step 109, where the digital sig- 
30 nature algorithm S is applied to the computed hash 
value , H, and the secret key, to compute the m-bit 
watermark W = S(H, KJ. 

[0037] Referring primarily to Fig. 3 and secondarily to 
Figs. 1 and 2, in the second pass of the watermark in- 

35 sertion procedure, the input JPEG image I is once again 
fed to a decoder which parses its headers noting the 
value of q, the quantizer for the 63 rd coefficient, in step 
201 . The headers for the watermarked output JPEG im- 
age, l w , are produced in step 202 by copying them di- 

40 rectly from the input image, except that the quantizer of 
the 63 rd coefficient in the quantizer table is changed to 
be 1 instead of its previous value, q. If there is another 
block of coefficients yet to be decoded and processed 
(step 203), the next such block, &,, is partially decoded 

45 jn step 204. Again, only the entropy coding of the com- 
pressed data is undone, avoiding the de-zig-zagging, 
dequantization, and IDCT steps needed for full decom- 
pression. This results in a representation of Bj made up 
of only the non-zero quantized coefficients (except for 

50 the 63 rd coefficient which is always included in the rep- 
resentation) along with their locations in the zig-zag or- 
der. The 63 rd coefficient of each block is multiplied by 
the q, in step 205. EMBEDDER-TEST is performed in 
step 206 to determine whether block Bj is supposed to 

55 embed the next bit of W. This determination may be 
made again on a block-by-block basis or may be made 
using the results of the previous decision-making proc- 
ess (step 105), if those results are stored in memory. In 
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any case, if block Bi is supposed to embed the next bit 
of W, then the LSB of the 63^ DCT coefficient of B, is 
set to match the next bit of W in step 207 and the pro- 
cedure proceeds to step 208. If the decision in step 206 
is "no", then the procedure directly proceeds to step 208. 
In step 208, the coefficients in B t are encoded and pro- 
duced as output into the compressed data stream for 
the watermarked image, l w . Note that the representation 
of the quantized coefficients of B s that we use enables 
efficient encoding as the quantized coefficients are al- 
ready in the zig-zag order, thus avoiding the DCT, quan- 
tization, and zig-zagging steps generally required for 
compression. The process repeats until all of the blocks 
have been processed. 

[0038] The verification procedure for the frequency 
domain watermarking method is illustrated in Figs. 4 and 
5 and is used to determine if the image has been tam- 
pered with. Referring to Fig 5, after initializing the hash 
value H in step 301 , the watermark verification proce- 
dure begins to decode the input image, l w , parsing its 
headers, in step 302. In step 303, it is determined wheth- 
er another biock remains to be decoded. If so, the next 
block, Bj, is partially decoded in step 304. Again, only 
the entropy coding of the compressed data is undone, 
avoiding the de-zig-zagging, dequantization, and I DCT 
steps needed for full decompression. This results in a 
representation of Bj made up of only the non-zero quan- 
tized coefficients (except for the 63 rd coefficient which 
is always included in the representation) along with their 
locations in the zig-zag order EMBED DER-TEST is per- 
formed in step 305 to determine whether block Bj is sup- 
posed to embed the next bit of W. If it is, then the LSB 
of the 63 rd coefficient of Bj is extracted as the next bit of 
the watermark W in step 306, and then that LSB is set 
to zero in step 307. The procedure moves to step 308 
from step 307 as well as from step 305 when the block 
Bj is not a watermark bit embedder. The hash value H 
is updated in step 308 using its previous value, and the 
values of all the non-zero quantized coefficients in Bj 
along with their locations in the zig-zag order, using the 
one-way hash function. The 63 rd coefficient is always 
included in this computation even if it is zero. The proc- 
ess continues through all the blocks and in the end, the 
extracted watermark Wand the hash value H have been 
fully computed. At this point the digital signature verifi- 
cation algorithm V (corresponding to the signing algo- 
rithm S) is applied using the public key K p (correspond- 
ing to the secret key K s ) to verify whether or not W is the 
same as S(H, K s ) in step 309. Note that step 309 does 
not require the use of the secret key frC,. Step 310 ex- 
amines the output of the digital signature verification al- 
gorithm V(W,H,Kp) applied in step 309. If the verification 
algorithm V(W,H,K p ) is successful, then the image has 
the fragile watermark intact, and thus has not been tam- 
pered with (step 312). If V(W,H,K p ) outputs failure, then 
the fragile watermark (if it ever existed) has been de- 
stroyed, and it may be concluded that the image has 
been tampered with or never had the fragile watermark 



corresponding to the key pair (K^Kp) (step 311). 
[0039] The effectiveness of the image tampering de- 
tection process depends on the strength of the hash 
function and the signing and verification algorithms S 

5 and V. A variety of one-way hash functions may be used, 
such as a hash function called MD5 developed by R. 
Rivest, or the SHA, or RIPEMD hash functions. Similar- 
ly, there is considerable choice available for the signa- 
ture and verification algorithm pair, such as the El Gamal 

10 scheme, the DSA algorithm, or the RSA algorithm. 
[0040] This fragile watermarking procedure can be 
modified to detect roughly the area of the image in which 
some tampering has been done. This is accomplished 
by dividing the image into some number of regions and 

75 applying the whole insertion procedure separately on 
each region. Only the regions that contain tampering will 
show a damaged watermark. This modification will not, 
however, detect a restricted form of tampering which is 
done by forming an image that is a collage of several 

20 regions extracted from different images or reordered re- 
gions from a single image, each region carrying a valid 
watermark. Such tampering, however, is likely to be vis- 
ually obvious, if the regions are large enough. 
[0041] Similar fragile watermarking techniques can 

25 be applied in the spatial domain as well. Instead of the 
highest-frequency coefficients, all or some of the pixels 
can be directly used as watermark bit embedders by set- 
ting their LSB to zero prior to the hash computation and 
then setting that LSB to the watermark bit in the second 

30 pass. Note that this invention does not suggest intermin- 
gling of the spatial and frequency domain watermarking 
processes. It does not suggest that watermark insertion 
can be done in the frequency domain as revealed in this 
invention while the corresponding watermark verifica- 

35 tion can be done in the spatial-domain as revealed in 
this invention. 

[0042] In addition, the watermark can be chosen to be 
a visible watermark. A visual watermark can be embed- 
ded in the frequency domain by computing the transform 

^0 frequency coefficients of a distinctive watermarking sig- 
nal, and simply adding the coefficients to those of the 
image. This process works as a result of linearity of the 
transforms commonly used in image coding (such as 
DCT), which ensures that addition in the pixel domain 

45 corresponds to addition in the frequency domain. A sim- 
ilar visible watermarking process can be performed in 
spatial-domain as well. 

[0043] It should be noted that the block and flow dia- 
grams used to illustrate the watermark insertion, extrac- 

so tion and verification procedures of the present invention, 
illustrate the performance of certain specified functions 
and relationships thereof. The boundaries of these func- 
tional blocks have been arbitrarily defined herein for the 
convenience of description. Alternate boundaries may 

55 be defined so long as the specified functions and rela- 
tionships thereof are appropriately formed. Moreover, 
the flow diagrams do not depict syntax or any particular 
programming language. Rather, they illustrate the func- 
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tional information one skilled in the art would require to 
fabricate circuits or to generate software to perform the 
processing required. Each of the functions depicted in 
the block and flow diagrams may be implemented, for 
example, by software instructions, a functionally equiv- 
alent circuit such as a digital signal processor circuit, an 
application specific integrated circuit (ASIC) or combi- 
nation thereof. 

[0044] The watermarking techniques of the present 
invention may be employed in connection with various 
devices including a digital camera, a block diagram of 
which is illustrated in Fig. 6. Operating under microproc- 
essor control, the digital camera 20 has a charge-cou- 
pled device (CCD) image sensor that captures an image 
and converts it to an analog electrical signal in block 21 . 
The analog signal is then processed and digitized in 
block 22, after which the digital image is temporarily 
stored in a frame buffer 23 while it undergoes digital 
processing in block 24. The digital image processing 
block 24 performs several functions including compres- 
sion and decompression. Processing block 24 may also 
perform the watermarking techniques of the present in- 
vention using hardware or software. Under user control 
25, the processing block 24 interfaces with in-camera 
image storage 26 where decompressed image data 
may be stored. The storage block 26 may comprise 
compact magnetic or solid-state storage media, either 
removable or fixed within the camera 20, and may in- 
clude removable, large-capacity PCMCIA-format hard 
disk cards or flash memory cards. 
[0045] The camera 20 includes analog and digital out- 
puts, 27 and 28 respectively, through which image data 
may be transmitted within the camera or to external de- 
vices. Uncompressed image data may be transmitted, 
via the analog outputs 27, to an LCD screen 29 within 
the camera 20, or to external devices such as a VCR or 
television receiver. Image data, whether compressed or 
uncompressed, may also be transmitted through the 
digital outputs 29 to a digital device such as a computer 
system where the image could be displayed or where 
watermarked images could be verified. 
[0046] Fig. 7 is a block diagram that illustrates the in- 
terrelationship between various components that may 
be used in capturing, processing and viewing digital im- 
ages. One of the more important components is a com- 
puter system, identified generally by reference numeral 
30. The computer system 30 may be of any suitable type 
such as a main frame or personal computer. 
[0047] Computer system 30 comprises a central 
processing unit (CPU) 31 which may be a conventional 
microprocessor, a random access memory (RAM) 32 for 
temporary storage of information, and a read only mem- 
ory (ROM) 33 for permanent storage of information. 
Each of these components is coupled to a bus 34. Op- 
eration of the computer system 30 is typically controlled 
and coordinated by operating system software. The op- 
erating system, which is embodied in the system mem- 
ory and runs on CPU 31, coordinates the operation of 



computer system 30 by controlling allocation of system 
resources and performing a variety of tasks, such as 
processing, memory management, networking and I/O 
functions, among others. 

s [0048] Also coupled to bus 34 by a controller 35 is a 
diskette drive 36 into which a non-volatile mass storage 
device such as a diskette 37 may be inserted. Similarly, 
a controller 38 interfaces between bus 34 and a compact 
disc (CD) ROM drive 39 which is adapted to receive a 

10 CD ROM 40. A hard disk 41 is provided as part of a fixed 
disk drive 42 which is coupled to bus 34 by a disk con- 
troller 43. 

[0049] Software for the watermarking techniques may 
be stored on storage devices and transferred to CPU 31 

is for execution. Alternatively, the software may be stored 
in RAM 32 or ROM 33. Similarly, image data be loaded 
into and extracted from computer system 30 using re- 
movable storage media devices such as the diskette 37 
and CD ROM 40. 

20 [0050] Image data may be input into computer system 
30 in other ways as well. Film-based images 44 gener- 
ated by a film camera 45 can be digitized by a scanner 
46 for storage and processing by the computer 30. The 
digital camera 20 can directly digitize images and trans- 

25 mit them to the computer 30, as explained above. A key- 
board 51 and mouse 52, which are coupled to bus 34 
via a controller 53, facilitate the input of such data and 
otherwise provide a means for entering information into 
computer system 30. 

30 [0051] Image data may also be transferred to and 
from computer 30 for remote locations. To this end, com- 
puter 30 may also include a communications adapter 54 
which enables the computer 30 to communicate with 
networks 55, which may include local area networks 

35 (LANs), the internet or online services, via direct con- 
nections or via modem. 

[0052] In accordance with the invention, images that 
have been previously watermarked, say, in the digital 
camera 20 may be transmitted to computer 30 for veri- 
40 fjcation. Alternatively, unmarked images may be water- 
marked and later verified in computer 30 using appro- 
priate hardware or software that is executed by the CPU 
31. 

[0053] Digital images transmitted or stored in compu- 
45 ter 30 may be viewed in a number of different ways. A 
printer 56 attached to computer 30 can produce color 
prints that vary in quality depending on the printer 56. 
Another option is to view the images on a display 57 
associated with the computer 30. Yet another choice is 
50 to display the images on a television monitor using a 
VCR. 

[0054] While the invention has been described in con- 
junction with specific embodiments, it will be evident to 
those skilled in the art in light of the foregoing description 
55 that many further alternatives, modifications and varia- 
tions are possible. For example, visible watermarking 
process and the invisible watermarking process de- 
scribed in this invention can be combined depending up- 
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on the application. Furthermore, the choice of blocks to 
be used in the watermarking process can be made ap- 
plication dependent. Thus, the invention described 
herein is intended to embrace all such alternatives, 
modifications, applications and variations as may fall 
within the scope of the appended claims. 



Claims 

1 . A method of watermarking a compressed digital im- 
age, comprising the steps of: 

partially decoding the compressed digital im- 
age to generate a plurality of data blocks, each 
block having a plurality of transform coeffi- 
cients; 

determining whether to embed a watermark bit 
in each block based on one of the plurality of 
transform coefficients in that block and a 
number of watermark bits remaining to be em- 
bedded in the digital image; 

computing a watermark, having a plurality of 
bits, for the entire digital image by applying a 
digital signature algorithm and a secret key; 
and 

embedding a watermark bit in each block for 
which it was determined to do so in said deter- 
mining step by setting the one of the plurality of 
transform coefficients in that block to match a 
corresponding bit of the computed watermark. 

2. The method of claim 1 , wherein the one of the plu- 
rality of transform coefficients in each block is the 
transform coefficient representative of the highest 
vertical frequency and highest horizontal frequency 
in that block. 

3. The method of claim 1 , wherein said step of com- 
puting the watermark further comprises the step of: 

setting to zero at least one bit of the one of the 
plurality of transform coefficients in each block 
for which it is determined in said determining 
step to embed a watermark bit 

4. The method of claim 3, further comprising the step 
of: 

encoding the plurality of blocks after said em- 
bedding step is completed to re-compress the 
watermarked image. 

5. The method of claim 3, wherein said of computing 
the watermark further comprises the step of: 



70 



updating a hash value at each block by applying 
a hash function at each block, wherein the hash 
value computed at a last block is the multiple- 
bit hash value representative of the entire dig- 
ital image. 

The method of claim 5, wherein said step of com- 
puting the watermark further comprises the steps 
of: 

obtaining a quantizer of the one of the plurality 
of transform coefficients in each block; and 



multiplying the one of the plurality of transform 
is coefficients in each block by its quantizer. 

7. A method of watermarking a compressed digital im- 
age, comprising the steps of: 

20 computing a watermark, having a plurality of 

bits, for the entire digital image, comprising the 
steps of: 

partially decoding the compressed digital 
25 image to generate a plurality of data 

blocks, each block having a plurality of 
transform coefficients; 

obtaining a quantizer of the one of the plu- 
30 rality of transform coefficients in each 

block; 

multiplying the one of the plurality of trans- 
form coefficients in each block by its quan- 
35 tizer; 

determining whether to embed a water- 
mark bit in each block based on one of the 
plurality of transform coefficients in that 
40 block that is representative of a highest 

vertical frequency and a highest horizontal 
frequency in that block and a number of 
watermark bits remaining to be embedded 
in the digital image; 

45 

setting to zero at least one bit of the one of 
the plurality of transform coefficients in 
each block for which it is determined in said 
determining step to embed a watermark 
so bit; 

updating a hash value at each block by ap- 
plying a hash function at each block, 
wherein the hash value computed at a last 
55 block is a multiple-bit hash value repre- 

sentative of the entire digital image; and 

applying a secret key and a digital signa- 
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ture algorithm to the multiple-bit hash value 
to compute the watermark; and 

embedding a watermark bit in each block for 
which it was determined to do so in said deter- 
mining step by setting the at least one bit of the 
one of the plurality of transform coefficients in 
that block to match a corresponding bit of the 
computed watermark. 

8. A method of detecting tampering of a compressed 
watermarked image, comprising the steps of: 

partially decoding the compressed water- 
marked image to generate a plurality of data 
blocks, each block having a plurality of trans- 
form coefficients; 

determining each block in which a watermark 
bit is embedded; 

extracting, from each block tor which it was de- 
termined in said determining step to be an em- 
bedder of a watermark bit, at least one bit of 
one of the plurality of transform coefficients in 
that block to generate an extracted watermark; 

computing a hash value of the digital image by 
applying a hash function at each block based 
on a zeroed value of the at least one bit of one 
of the plurality of transform coefficients in that 
block, wherein the hash value computed at a 
last block is a multiple-bit value representative 
of the entire digital image; and 

applying a digital signature verification algo- 
rithm to the extracted watermark and the mul- 
tiple-bit computed hash value and a public key 
to determine whether the compressed water- 
marked image has been tampered with. 

9. The method of claim 8, wherein the one of the plu- 
rality of transform coefficients in each block is the 
transform coefficient representative of the highest 
vertical frequency and highest horizontal frequency 
in that block. 

10. An image capturing device, comprising: 

a sensor for capturing light and converting the 
light into an analog image signal; 

an analog-to-digital converter for converting 
the analog image signal to a digital image; and 

a digital image processor for compressing the 
digital image to generate a plurality of data 
blocks, each block having a plurality of trans- 



form coefficients, determining whether to em- 
bed a watermark bit in each block based on one 
of the plurality of transform coefficients in that 
block and a number of watermark bits remain- 

5 ing to be embedded in the digital image, com- 

puting a watermark, having a plurality of bits, 
for the entire digital image, and embedding a 
watermark bit in each block for which it was de- 
termined to do so by setting the one oflhe plu- 

10 rality of transform coefficients in that block to 

match a corresponding bit of the computed wa- 
termark. 

11. The image capturing device of claim 10, wherein 
is said digital image processor sets to zero at least 
one bit of the one of the plurality of transform coef- 
ficients in each block for which it is determined to 
embed a watermark bit. 

20 12. The image capturing device of claim 11, wherein 
said digital image processor encodes the plurality 
of blocks after the watermark bit embedding is com- 
pleted to re-compress the watermarked image. 

25 13. The image capturing device of claim 11, wherein 
said digital image processor updates a hash value 
at each block by applying a hash function at each 
block, wherein the hash value computed at a last 
block is the multiple-bit hash value representative 

30 of the entire digital image. 

14. The image capturing device of claim 11, wherein 
said digital image processor obtains a quantizer of 
the one of the plurality of transform coefficients in 

35 each block and multiples the one of the plurality of 
transform coefficients in each block by its quantizer. 

15. A computer system including a processor and a 
memory having a computer-readable program code 

40 embodied therein for causing the processor to de- 
tect tampering of a compressed watermarked digit- 
al image by performing the steps of: 

partially decoding the compressed water- 
45 marked image to generate a plurality of data 

blocks, each block having a plurality of trans- 
form coefficients; 

determining each block in which a watermark 
50 bit is embedded; 

extracting, from each block for which it was de- 
termined in said determining step to be an em- 
bedder of a watermark bit, at least one bit of 
55 one of the plurality of transform coefficients in 

that block to generate an extracted watermark; 

computing a hash value of the digital image by 
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applying a hash function at each block based 
on a zeroed value of the at least one bit of one 
of the plurality of transform coefficients in that 
block, wherein the hash value computed at a 
last block is a multiple-bit value representative 
of the entire digital image; and 

comparing the extracted watermark with the 
multiple-bit value by applying a digital signature 
verification algorithm to determine whether the 
compressed watermarked image has been 
tampered with. 

16. The computer system of claim 15, wherein the one 
of the plurality of transform coefficients in each 
block is the transform coefficient representative of 
the highest vertical frequency and highest horizon- 
tal frequency in that block. 

17. A method of watermarking a digital image, having 
a plurality of pixels, comprising the steps of: 

computing a watermark, having a plurality of 
bits, for the entire digital image, comprising the 
steps of: 

determining whether to embed a water- 
mark bit in each pixel based on one of a 
plurality of bits representative of that pixel 
and a number of watermark bits remaining 
to be embedded in the digital image; 

setting to zero at least one bit of each pixel 
for which it is determined in said determin- 
ing step to embed a watermark bit; 

updating a hash value at each pixel by ap- 
plying a hash function at each pixel, where- 
in the hash value computed at a last pixel 
is a multiple-bit hash value representative 
of the entire digital image; and 

applying a secret key and a digital signa- 
ture algorithm to the multiple-bit hash value 
to compute the watermark; and 

embedding a watermark bit in each pixel for 
which it was determined to do so in said deter- 
mining step by setting the at least one bit of that 
pixel to match a corresponding bit of the com- 
puted watermark. 

18. A method of detecting tampering of a watermarked 
digital image, having a plurality of pixels, comprising 
the steps of: 

determining each pixel in which a watermark bit 
is embedded; 



extracting, from each pixel for which it was de- 
termined in said determining step to be an em- 
bedder of a watermark bit, at least one of a plu- 
rality of bits representative of that pixel to gen- 
5 erate an extracted watermark; 

computing a hash value of the digital image by 
applying a hash function at each pixel based on 
a zeroed value of the at least one bit of that pix- 
10 el, wherein the hash value computed at a last 

pixel is a multiple-bit value representative of the 
entire digital image; and 

applying a digital signature verification algo- 
15 rithm to the extracted watermark and the mul- 

tiple-bit computed hash value and a public key 
to determine whether the compressed water- 
marked image has been tampered with. 

20 19. The method of claim 1 7 or 1 9, wherein the at least 
one bit is the least significant bit. 

20. A method of watermarking a compressed digital im- 
age, comprising the steps of: 

25 

adding a visible watermark; 

computing an invisible watermark from the dig- 
ital image and the visible watermark, compris- 
30 ing the steps of: 

partially decoding the compressed digital 
image to generate a plurality of data 
blocks, each block having a plurality of 
35 transform coefficients; 

obtaining a quantizer of the one of the plu- 
rality of transform coefficients in each 
block; 

40 

multiplying the one of the plurality of trans- 
form coefficients in each block by its quan- 
tizer; 

45 determining whether to embed an invisible 

watermark bit in each block based on one 
of the plurality of transform coefficients in 
that block that is representative of a h ighest 
vertical frequency and a highest horizontal 

so frequency in that block and a number of in- 

visible watermark bits remaining to be em- 
bedded in the digital image; 

setting to zero at least one bit of the one of 
55 the plurality of transform coefficients in 

each block for which it is determined in said 
determining step to embed an invisible wa- 
termark bit; 
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updating a hash value at each block by ap- 
plying a hash function at each block, 
wherein the hash value computed at a last 
block is a multiple-bit hash value repre- 
sentative of the entire digital image; and s 

applying a secret key and a digital signa- 
ture algorithm to the multiple-bit hash value 
to compute the invisible watermark; and 

10 

embedding an invisible watermark bit in each 
block for which it was determined to do so in 
said determining step by setting the at least one 
bit of the one of the plurality of transform coef- 
ficients in that block to match a corresponding is 
bit of the computed invisible watermark. 



20 



25 



30 



35 



40 



45 



50 



55 



10 



EP 0 947 953 A2 



JPEG Image 



Secret key 
Ks 




Watermarked 
JPEG Image 
Iw 



Fig. 1 



11 



EP0 947 953 A2 



109 



Compute the watermark 
W by applying the digital 
signature algorithm S to 
H and the secret key Ks. 
W= S (H, Ks) 



No 



No 



f Start J 



Initianze H, 
the hash value 



i 



.101 



Start decoding the 
input JPEG image. I; 
set q- quantizer of 
63rd coefficient 



102 




104 



Partially decode the 
JPEG data to obtain 
the quantized OCT 
coefficients of the 
next block. Bi 



Multiply 63rd 
coefficient 
of Bi by q 



105 




Set LSBof63rd 
coefficient of 
Bi to zero 



108 



Update the hash value H 
by applying the hash 
function to its previous 
value and all the 
coefficients in Bi 



Fig. 2 

12 



EP 0 947 953 A2 



f Start ^ 



Start decoding the 
input JPEG image. I: 
set q=quantlzer of 
63rd coefficient 



201 



202 



Output the header for the watermarked 
Image Iw by copying the header of I, 
except changing of quantizer of 63rd 
coefficient from q to 1, 



Y 



No 




204 



Partially decode the 
JPEG data to obtain 
the quantized OCT 
coefficients of the 
next block, Bi 



Multiply 63rd 
coefficient 
of Bi by q 



205 



No 




Set LSB of 63rd coefficient 
of Bi to be equal to the 
next bit of W 



208 



Encode the coefficients in 
Bi and output as the next 
compressed block of Iw 



Fig. 3 



13 



EP 0 947 953 A2 



JPEG Image 
Iw 

Public key 

Kp 



Verify 
Watermark 



"Has watermark" 
or 

"Does not have watermark- 



Fig. 4 



14 



EP 0 947 953 A2 



Start 

— r 

Initialize H, 
the hash value 



^1 



309 



301 



Start decoding 
the input JPEG 
Image, Kv 



Apply the digital 
signature verification 
algorithm V on W.H 
and the public key, 
Kp:V{W. H. Kp) 



No 



310 



311 



\ 


[Yes 


Output "Does 
not have 
watermark* 


Output "Has 
watenriartc" 




I . I 









Stop 



1 




302 



303 



304 



Partially decode the 
JPEG data to obtain 
the quantized DCT 
coefficients of the 
next block. Bi 




Extract LSB of 63rd 
coefficient of Bi as 
the next bit ofW 



Set LSB of 63rd 
coefficient of 
Bi to zero 



S 



307 



Update the hash value H 
by applying the hash 
function to its previous 
value and all the 
coefficients in Bi 



308 



Fig. 5 



15 



EP 0 947 953 A2 






. CM 



N 




CO 

ri> 
ii- 



^3 



16 



EP 0 947 953 A2 



2 
O 






§« 




a: 


I 1 













in 




LL 



17 



This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 

□ FADED TEXT OR DRAWING 



td BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




